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Abstract 



A G/M/N queue is considered in a moderate deviation heavy traffic regime. The rate 
functional for the customers-in-system process is obtained for single class model. A risk- 
sensitive type control problem is considered for multi-class G/M/N model under moderate 
deviation scaling and shown that the optimal control problem is related to a differential 
£\J ■ game problem. 
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1 Introduction 

Studying scaling limit is an established tradition in queuing theory. These include heavy traffic 
approximation that depends on central limit theorem (CLT) and large deviation (LD) approx- 
imations. Another interesting scaling, considered in queuing network, is moderate deviation 
(MD) scaling which includes an intermediate scaling of CLT and LD. MD scaling are consider 
when the queuing network is critically loaded. Therefore MD can seen as LD analogue for 
heavy traffic set up. Also some control problems in MD regime have interesting characteristic 
that also appears in the asymptotic regime alluded to LD and heavy traffic approximations 

There have been several works on MD scaling without dynamic control aspect. LD and 
MD for renewal processes are proved in [17]. Later in |16j . Puhalskii obtains MD principle 
for queue length and waiting-time processes for single class single server network. Majewski 
[15j considers feedforward multi-class network with priority and obtains MD asymptotics for 
waiting time, idle time, queue length, departure and sojourn time processes. We refer to [19] , 
|10| . for various interesting aspects of MD regime. A dynamic control problem for multi-class 
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G/G/l queue in MD regime is considered in pQ where the authors point out some interesting 
features of the problem similar to other asymptotic regimes. 

So far MD asymptotics have not been considered in many server queuing network. In 
this article we introduce MD principle for the customers-in-system process in a many server 
network. We consider a single class G/M/N queuing network where the arrival is given by a 
general renewal process and the service requirements are exponential. We show that the rate 
functional for customers-in-system process in MD regime changes depending on the growth 
rate ./V compare to arrival rate A n . It is shown that if N = o(A n ) then the rate functional in 
MD regime for customers-in-system process is governed by Skorohod map. But if ^ A as 
n —> oo, the governing dynamics for the rate functional are not reflection maps. It is worthwhile 
to mention that this problem can be seen as MD analogue of the scaling considered by Halfin 
and Whitt for G/M/N queuing network in [12]. One may wish to consider the MD analysis 
for G/G/N network but the problem is harder as one needs to consider an infinite dimensional 
set up for the problem. 

We also consider a risk-sensitive type control problem for multi-class G/M/N network 
when N = o(X n ). We consider I different customer classes arriving to a parallel server system 
following I independent renewal processes. Service time distributions are exponential with 
class dependent parameters. Each customer is served by one of the servers and servers are 
not allowed to serve more than one customers at the same time. The problem is to control 
B n = (Bf, . . . , B^) where Bf denotes the number of class— i customers in service, so that the 
cost is minimized. Denoting by X™, the number of class— i jobs in the n-th system, the scaled 
version is given by X" = — ^ where pi denotes the limiting traffic intensity for class— i and 
lim6 n = oo, lim = 0. The cost is given by 

J_ E [ e ^(/o T h(X n (s)ds+g(X n (T)))i 

where T > 0, and h, g are given nonnegative functions. This risk-sensitive type of cost has 
been studied in literature for its own importance (see [2], [3], [18]). One of the important aspect 
of the exponential cost is that it penalizes large quantities heavily. This is one of the reason for 
considering exponential cost attached to the queue length or customers-in-system processes. 
Another interesting aspect of working in MD regime is that the limiting differential game (DG) 
is solvable pQ. 

It is also interesting to compare the control problem above with the existing similar control 
problems ([I], [3]). In [3], the authors consider a similar problem (with bounded h) for multi- 
class M/M/N network in LD regime. The convergence of the value function, corresponding to 
above optimal control problem, is proved using martingale method. In pQ, a similar problem 
is considered for multi-class G/G/l network and the convergence result is obtained by con- 
structing a particular policy. In both the problems, the servers are allowed to serve more than 
one customers simultaneously. First of all, our proof technique here does not use any PDE 
analysis like [3]. Also in our above described model, we do not allow processor sharing. So 
the set of controls considered in this paper is smaller than those that are considered in earlier 
works. The proof of the convergence of the value function for the optimal control problem is 
divided into two parts. We first prove the lower bound estimate following similar technique 
as PQ . The proof for the upper bound is based on the construction of a particular policy such 
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that the lower bound is asymptotically attained. The construction of this policy is compli- 
cated than that appear in pQ and can be used to improve the control set used in pQ. We also 
obtain a simple control when the cost functions are linear and N = o{b n y/n). [61 U] deal with 
a multi-class G/M/N network under diffusion scaling where N ~ y/n. Our problem can also 
be thought of as a generalization to these works in risk-sensitive set up. Let us also mention a 
related work [5] where a multi-class scheduling problem is considered under diffusion scaling. 

To summarize the main contribution of the paper, we have (a) introduced the moderate 
deviation scaling for many server queues in heavy traffic regime, (b) shown the convergence of 
value function for optimal control problem to a value function of DG, (c) considered a smaller 
class of admissible control which can also be used to improve the results in pQ, (d) given a 
simple policy when the cost functions are linear and N = o(6 n y / n). 

Notations: For a positive integer k and a, b € R k , a - b denotes the usual scalar product, while 
|| • || denotes Euclidean norm. For a £ R+, [a\ denote the largest integer less than or equal 
to a. Given a, b S R, the maximum (minimum) is denoted by a V b (a A b). We use a + (a~) 
for a V (—a V 0). Given two sequences {a n }, {b n }, a n = o(b n ) means limsupl^ = 0. By 
K^L we denote the nonnegative orthant of the Euclidean space R k . For T > and a function 
/ : [0,T] -> R k , let ||/||* = sup se[M t G [0,T]. When k = 1, we write \f\* t for 

and ll/H* for ||/||*n. e(-) is used to denote the identity function on R. For T < oo, denote by 
C([0, T], R k ) and D([Q, T],R k ) the spaces of continuous functions [0, T] — > R k and, respectively, 
functions that are right-continuous with finite left limits (RCLL). For fix T > 0, endow the 
space D([0,T],R k ) with the Skorohod-Prohorov-Lindvall metric or J\ metric, defined as 

dfa y') = inf (||/|r V sup \\<p(t) - <f/(f(t))\\) , <P, V' G D([0, T],R k ) 
where T is the set of strictly increasing, continuous functions from [0, T] onto itself, and 

/w - m 



sup 

0<s<t<T 



log 



t - S 



As is well known [7], D([0,T],R k ) is a Polish space under the induced topology. Through 
out this article, we fix a complete probability space (f2, ¥). All the stochastic processes 
introduced in this paper are defined on (f2, J 7 , P). 

The paper is organized as follows. The next paragraph introduces some preliminaries that 
will be used in this paper. Section [2] introduces moderate deviation principle for a single class 
G/M/N queue. Section [3] is devoted to the study of the multi-class G/M/N queues and the 
dynamic control problem. Section [3J] introduces the associated dynamic games and states the 
main results. The proof of the main results are given in Section [3.2i Finally, in Section [3.31 we 
prescribe a simple control which is asymptotically optimal when the cost functions are linear. 



Preliminaries: Now we state the definition and properties of large deviation principle (LDP) 
and Skorohod problem that will be used in this paper. Given a metric space S, a function 
I, defined on S, is said to ba a rate function if the set {x £ S : l(x) < a} is compact for 
all a > 0, and there is a sequence {P n }n>i of probability measure on the Borel cr-field of S 
(or sequence of random variable {^"} with law P n ) satisfying large deviation principle with 
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parameter a n — > oo and rate function I i.e., 



limsup — logP n ,(F) < — inf I(x) 
an xeF 



for all closed set F C S, and 

limsup — logP n (F) > — inf I(x), 

for all closed set G C S. 

On standard way to get new LDP's from an existing one is through contraction mapping 
principle which states that if {^ n } satisfies LDP with rate function I and / is a continuous 
function on S, then f(X n ) satisfies LDP with rate function 

I f (y)= mf I(x). (1.1) 

There are several extension to this contraction mapping principle. We refer to jll] for a survey 
on contraction mapping principles. In this article, we use an extended contraction mapping 
principle which states that if {X n } obeys LDP with rate function I, {f n } is a sequence of 
measurable functions, and if there is a measurable function /, continuous when restricted to 
the set {x : I(x) < a}, a > 0, and f n {x n ) — > f(x) as n — > oo whenever x n — > x and I(x) < oo, 
then {f n (X n )} n >i obeys LDP with rate function given by (jl.ip . 

Our goal in this paper is to study asymptotics of certain value functions and to show that 
they lead to the value function of certain differential game problem. This differential game 
problem is solvable. In order to define the solution to the game we need do define Skorohod 
problem. 

Definition 1.1 Let tp G D([0, oo), M) with ip(0) G R+ be given. Then (<p l ,(j) 2 ) solves the 
Skorohod problem for the data ip if ip(0) = </ )1 (0), and for all t G [0, oo) 

1. 4>\t) = m+<p 2 {t), 

2. <f>\t) G R+, 

3. (p 2 is nondecreasing, 

I J °V(s)# 2 (s) = o. 

It is know that the above problem has a unique solution (|13j. [8]). Define T(V') = 4> l - ^ is 
referred to as Skorohod map. In fact, V has an explicit form given by 

r(v>)(t) = m + sup 

0<s<t 

It is easy to see that T satisfies Lipschitz property i.e., 

|r(^ 1 )-r(v 2 )|^<2|^ 1 -v 2 lT, 

for ^ G D([0, oo), R), ^(0) G R+, i= 1, 2. 
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2 Moderate deviations for many server queues 

In this section, we introduce a single class G/M/N model. We consider a parallel server 
system with single customer class and a pool of identical servers. Let A n be given parameter 
where represents the mean of the inter-arrival times of customers in the n-th system. 
Let {IA{1) : I G N} be a given sequence of i.i.d. of positive random variables with mean 
E[.L4(1)] = 1 and variance Var(-L4(l)) = o~f IA . Assuming Yli = 0; the number of arrivals of 
customers up to time t, for the n-th system, is given by 

A»(*)=sup{j>0 

k=l 

N n G N denotes the number of servers in the n— th system. Service time distributions are 
exponential. Let /i n be the rate at which customers are served in the n-th system. We also 
consider the moderate deviation rate parameters {b n } with the property that lim6 n = oo while 
lim -^8= = 0. We assume that as n — > oo, 

> A > 0, — -> u > 0, ^— — ) — t- r G (-oo, oo). 2.1) 

n n b n n n 

It is easy to see that under (|2.ip . A = u i.e., the system is critically loaded. A similar condition 
in [16] is referred to as near-heavy-traffic condition. 

Let X n denote the number of customers in the system. Let S(-) be a standard Poisson 
process independent of the arrival process. The number of service completions of jobs by time 
t is given by 

D n (t) = S(ji n /V(«)), (2.2) 
J o 

where Z n denote the number of customers in service. Hence we have 

X n (t) = X n (0) + A n {t) - D n (t). (2.3) 

The system is assumed to work under non-idling policy i.e., Z n = X n A iV n . Next we define 
the scaled process as follows 

A n {t) = —^(A n (t)-\ n t), S"(t) = —^(S(N n n n t)-N n Li n t), X n {t) = -^(X n {t)-N n ). 
b n V n Kyn b n y/n 

(2.4) 

It is easy to see from (12. 3|) that 

1 N n n n fri 1 

X n (t) = X n (0) + y n t + A n (t) - S^—T n (t)) + —^~^(t ~ J^T n {t)) 

1 N n u n n I' 1 ~ 

= X n (0) + y n t + A n {t) - S;(—T n (t)) + —^-j^ J o (X n (s))-ds, (2.5) 

where y n = - ^-), T n {t) = /„* Z n {s)ds. We fix T > and assume: 

Condition 2.1 The process (A n ,S™) satisfies large deviation principle (LDP) in D([0, T], M 2 ) 
with parameter b\ and rate function I taking value oo on discontinuous paths. 
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Remark 2.1 Because of independence, it is enough if the processes A n ,S™ satisfy LDP indi- 
vidually. In fact, one can impose some sufficient conditions on the inter-arrival processes so 
that Condition \2. 1\ holds (see Remark \3.S\ and \3.3\ below). 

We also assume that the initial condition is deterministic and 

l"(0)^iel, as n^oo. 

We are interested to find the rate function for X n . We subdivide the problem in two theorems. 

Theorem 2.1 Assume Condition \2.1\ holds and N n = n. Then {X n } n >i defined in (|2.4p 
satisfies LDP in D([0,T],M) with parameter b\ and rate function Ix given by 

lxW= mf IW^ 2 ), 

where Gfy 1 ,^ 2 ) denotes the solution to the equation 

^ = x + r t + ^ l (t)-i) 2 {t) + {X [ (ip(s))~ds. (2.6) 

Jo 

Proof: From (|2.5p . we have 

1 N n u n f l ~ 

X n {t) = X n (0) + y n t + A n (t) - S;(—T n (t)) + —0- / (X n (s))-ds. (2.7) 

JV n Jo 

Now given any tuple (x, y, tz, tp , ip 2 ) £ M 3 x D([0,T],~M?) it is easy to see that there exists a 
unique £ € D([0,T],R) satisfying the following: 

^(t) =x + yt + i; 1 (t)- tf{t) + k f {£{?))- ds, (2.8) 

J o 

m<e KT (\x + yt + ^(t) + ^(t)\* T ). (2.9) 
Since (A n , S^) satisfies LDP with rate function I and {I < a}, a > 0, is compact, we have 

lim limsupP^(|i n |^ + |Sft|rl > <*) = 0. (2.10) 

Now for any 5 > 0, 

p (l* " ^"OIt > 8) < He^ T \X n (0) + y n t +A n - <^(_r^))|^ > 
where we have used (|2.9p . Therefore using (|2.ip and (|2.1Up . we have 

hmsu P p4(|t - -}-T n (t)\* T >5) = 0. (2.11) 

rn>oo J' 
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Hence the sequence {i^-T 71 } converges super-exponentially in probability at rate -X- to e(-) in 
D([0,T],R) where e(t) = t. Therefore (i n , 5™ o (^T n )) satisfies LDP with rate function I 
in D([0,T],R 2 ) ([T7|. Lemma 4.3). Denote £ by G(x, y, ip 1 , 4> 2 , k) where £ satisfies ([IS]). Let 
-> (V^ 1 ,^ 2 ) for some continuous path (V^ 1 ,^ 2 ). Let £ n = G(X n (0), y n , V£, ^ 2 , K n ), 
K n = ^ , and £ = G(x, r, ?/> 2 , /i). Then it is easy to see that |£ n — £|y — > as n — ^ oo. 
Therefore extended contraction mapping principle yields that X n satisfies LDP with parameter 
b\ and rate function 

IxW= inf I$\4> 2 ), 

where G^ip 1 ,^ 2 ) denotes the solution to (|2.6|) . □ 

Theorem 2.2 Assume Condition \2.1\ holds. Let N n = o(n) and x G R + . T/ien {X n } n >i 
defined in (|2.4p satisfies LDP in D([0,T],R) uiift parameter b\ and rate function Ix given by 

I x (iP) = inf _ I^ 1 ,^ 2 ), 

?/)=r(x+j/e+)/j 1 — f/> 2 ) 

where T(-) denotes the Skorohod map. 
Proof: From (|2.5p . we have 



1 7V n ;v n n P t ~ 

X n {t) = X n (0) + y n t + A n {t) - S;(—T n (t)) + — f-^ / (X n (s))-ds. (2.12) 

J 

By our assumption on N n , we have — >• oo as n — > oo. Given 5 > 0, we define the (5-oscillation 
function osc«5 : D([0, T],R) — >■ R + as follows: 

oscs(ip) = sup{\ip(t) — ip(s)\ : |t — s| < <5}. 

Applying the extended contraction mapping principle we see that (oscs(A n ), osc$(S™)) satisfies 
LDP in Rj_ with rate function Iq given by 

|0 far (*,») = (0,0), 
| oo otherwise. 

Therefore, given any 5\ > 0, 

limsupp4( OSC5 (i n ) + os C(5 (S™) > <5i) = 0. (2.13) 

n— >oo 

Now choose e > 0. We claim that 

limsupP^(|(X n )"|r > 3e) = 0. (2.14) 

71— too 

Define Q n := {|(X n ) _ |j, > 3e}. Choose n large enough so that X n (0) > — §. For each w 6 fi n , 
we will have random times < a\ < a% < T such that X n (af) > -f ,X n (a%) < , and 
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X n (s) < —e on [c™,^]. This is possible to do as the jump size of X n is b K^ . Hence from 
(j2TT2|) . we have 

N n u n n 

-e > y n (a% - - osc^{A n ) - osc ff «_ ff «(^) + —^—e(*% - <). (2.15) 

Now if (o§ - a?) > Si for some fix 5i > then (j2J5]) implies that 2(\y n \T + \A n \* T + |S»|* ) > 
^^-ffneSi. If (of - O < <5i then + osc 5l (i n ) + osc 5l (5") > e. Therefore if we 

choose Si > so that |y n <5i| < | for all re large, then 

H\{X n )-\*T > 3e) < P(|A»|J + |5J|J > «(— )^) +P( 0SC5l (A") + 0SQl (^) > £ /2), 

where — )■ oo as n — >• oo. Therefore using (|2.10j) and (|2.13[) . the claim f)2. 14j) follows. 
Now rewriting (|2.12j) as 

1 N n u n n f f ~ 

(X n (t)) + = (X n (t)r+X n (0)+y n t+A n (t)-S;(— T(t)) + — ^— / (2.16) 

J 



we see that (X n (t)) + solves Skorohod problem for the date {X n (t))~ + X n (0) + y n t + A n (t) 
S]l(^nT n (t)). Hence using the Lipschitz property of the Skorohod map we have 

N n n n n I" ~ 

I r* I ^ vra/ „\\— j„i* ol / vn fj.\\ — i vn/n\ i „.»i-i i ^nw rrm/ 



- jf < 2\(X n (t)r + X n (0) + y n t + A n (t) - s;{-^T n {t))\* T . (2.17) 



n N' 

1 rpn/j-W n ft ( y 



Since ^(i - ^rT n (t)) = f c t {X n (s))- ds, applying (|2li)j) . (plij) and (I2T7D . we have 



limsupP#(|t- — T n (t)|^ > 5) = 0. 

Hence (A n ,S™ o (j^T n )) satisfies LDP with rate function I ([TTj, Lemma 4.3). Now we 
consider a sequence (V'n'V'n) ~~ ^ (V ;1 )' ( /' 2 ) as n — ^ oo for some continuous path (t/' 1 ,^ 2 ) £ 
D([0,T],IR 2 ). Let £ n be the solution to (ESD with the data (X n (0), y n , ^f-j^, fy, ^ 2 ). Let 
£ = T(x + ye + tp 1 — i)j 2 ). To complete the proof it is enough to show that |£ n — £|?p — > as 
n — > oo. The proof will follow from the extended contraction mapping principle. Given e > 0, 
we choose 5 > such that (usiip^+oosi^n)) < f f° r an n l ar g e - Since sup^lV^l^r + l^ 2 |y) < oo, 
we can choose 5 small enough to conclude that 

l(n-|T<3e, 

for large n (using (|2.15p ). From (|2.8p . we note that (£ n ) + satisfies Skorohod problem for the 
data (£ n (-)) _ +X n (0) + y n e{-) + %(•) — V> 2 (-) an d therefore Lipschitz property of the Skorohod 
map implies 

|(H + ~C\t < 2|(f (•))" + X n (0) + y»e(-) + #(•) - - (s + re(-) + ^(-) - 

< 8e, 

for all n large. Hence |£ n — £| ^ — > as n — > oo. This completes the proof. □ 
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3 Control of multi-class G/M/N 



In this section, we introduce a multi-class G/M/N model and a related control problem. We 
consider a parallel server system with I number of customer classes and a pool of identical 
servers. Let X = {1,2,..., I}. Let A" > 0,n £ N,i £ I, be given parameter where 4r 

i 

represents the mean of the inter-arrival time of class- i customers in the n-th system. Given are 
I independent sequence {I At (I) : I £ N}igi of positive random variables with mean E[7Aj(l)] = 
1 and variance Var(_L4j(l)) = of IA . Assuming J2i = 0' the number of arrivals of class-i 
customers up to time t, in the n-th system, is given by 

k=l 1 

N n G N denotes the number of servers in the n— th system. Service time distributions are 
exponential, with class dependent parameter. Let fj,f be the rate at which class— i customers 
are served in the n-th system. We also consider the moderate deviation rate parameters {b n } 
with the property that \imb n = oo while lim^oo V j^} = 0. Note that N n = o(n). We 
assume that as n — > oo, 

• it ^ Ai > and -)• m > 0, 

Hence the traffic intensity for class-*, namely N n^ n , has limit pi := The system is assumed 
to be critically loaded i.e., Yli=i Pi = 1- 



Let B™(t) be the number of servers working on class-i customers at time t > 0. Therefore 
B n = (Bf , . . . , BV" ) takes value in N . Let X", Q", I n denote the number of class- i customers 
in the system, the queue length of class-i customers in the buffer and the number of servers 
that are idle, respectively. Hence we have 



X? = Q? + B?, iel, (3.1) 

N n = I n + J2 B i- (3-2) 

We are given I independent standard Poisson processes Si,i £ I. The number of service 
completions of class— % jobs by time t is given by 

A n W = Sii^mt)), (3-3) 

where 

Jf= f B?{s)ds. (3.4) 
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Hence we have 

X?(t)=X?(0)+A?(t)-Dm- (3-5) 
For simplicity, the initial condition X n = (Xf, . . . , Xr) is assumed to be deterministic. The 
processes A n ,X n ,Q n ,B n will always be assumed to have RCLL sample paths. We will also 
assume that the processes Af,Si,i € I, are mutually independent. 

The process B n is regarded as control, that is determined based on the observation from 
the past (and present) events in the system. Fix T > 0. Given n, the process B n is said to be 
an admissible control if its sample paths lie in D([0, T], R+) and 

• B n {t) G N 1 for all t > 0; 

• For i £ X and t > 0, 

£r(i)<Xf, and J>f(i)<iV"; (3.6) 

• It is adapted to the filtration 

a{A?(s),D?(s),iel,s<t}. 

Denote the class of all admissible controls B n by il n . We can see that under admissible control 
each server is allowed to serve a single customer at a time. We do not allow processor sharing. 
Next we introduce the scaled processes. For i £ I, let 

Am = tA=(4- (*) - W), = -i_(s*(jv>?*) - N n tft), 

bnV n b n yjn 

X?(t) = ^(X?(t)-fHN»). (3.7) 



It is easy to check from f)3.5j) that 



Z n (t) = _£LX1(^ _ — 2?(t)), y? = A™ - (3.9) 



Xf (t) = Xf (0) + tft + AUt) - S n ^T n (t)) + Z?(t), (3.8) 

where we denote 

N^Vn (i 1 
n b n KP N 

Since X^ej — -A 7 "™ and ^ = 1, we see that 

starts from zero and is nondecreasing, (3.10) 

N n u? 1 

The initial condition X™(0) is assumed to satisfy the following: 

X™(0) -> x e R+, as ra -> oo. 

The scaled arrival processes ^4 n is assumed to satisfy a moderate deviation principle. Let us 
first define the rate functions. Let Ik,k = 1,2, be functions on D([0, T], R 1 ) defined as follows. 
For^ = (^i,...,Vl)G J D([0,T],M I ), 



I Si=i T~^2 — Jo* ''Pii^ds if all Y'i are absolutely continuous and -0(0) = 0, 
oo otherwise, 
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and 



h ^2i=i JT Jo i^f{s)ds if all ipi are absolutely continuous and tp(0) = 0, 
oo otherwise. 



Condition 3.2 (Moderate deviation principle) The sequence A n = (A™, . . . , A™), satisfies 
the LDP with parameters h\ and rate function I± in D([0, T], R 1 ); i.e., 

• For any closed set F C DQO^R 1 ) 

limsup-UogPfi™ G F) < - inf IM, 

tpeF 

• For any open set G C DQO^R 1 ) 

liminf -^logP(I n G G) > - inf l x M>). 

Remark 3.2 It is shown in \17f that each one of the following statements is sufficient for 
Condition \3.2\ to hold: 



• There exist constants oq > 0, /3 G (0, 1] such that E[e a °^ ] < oo , i G I, and 

• For some 5 > 0, E[(IAi) 2+s ] < cc, i G I, and b~ 2 log re — > oo. 

Remark 3.3 Since the inter-arrival time for a Poisson process is exponential, using Remark 
\3.S[ we see that S™ = (<S™ , . . . , S^), satisfies Large deviation principle in D([0, T],R : ) with 
parameter h\ and rate function I2. Therefore using the independence of the processes (see 
and extended contraction mapping principle we see that (A n , £*") satisfies Large deviation 
principle in D([0, T], R 21 ) with parameter 6 2 and rate function I(V') = Iify 1 ) ~\~^-2{' l P 2 ), V 7 = 
(^,^)6D([0,T],1 21 ). 

To present our control problem, we consider nonnegative functions h and g from R 1 to M 
which are nondecreasing with respect to the usual partial order on R 1 . We assume that h,g 
have at most linear growth, i.e., there exist constants C±,C2 such that 

g{x) + h{x) <Ci\\x\\+C 2 . 

Given n, the cost associated with the initial condition X n (0) and control B n is given by 

e blU£ h{X«-{s))d 8 + g {X n {T))\ 



We are interested to analyze the value function 



VJ(X w (0))= B jrf i JJ(X»(0) ) S»). 
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We now introduce another value function associated to the queue length. To do this, we define 

= bi^Q^ i G x ' and Q n = (<??. • • • > Qi)- Let 

e bllfo h(Q n (s))ds+g(Q n (T))] _ 
The associated value function is given by 

W"(o)) = j^j^o),^)- 

3.1 A differential game and main results 

We next develop a differential game for the limiting behavior of the value functions defined 
above. This game problem has been studied in pQ. Let 9 = (— , . . . , — ) and y = (yi, . . . , yj) 

where yi = \i~ pip*. Denote P = C ([0, T], K 21 ) (the subset of C([0, T], R 21 ) of functions with 
initial value 0) and 

E = {( G C([0,T],M I ) : 9 ■ ( starts from zero and is nondecreasing}. 

The topology on both the spaces are induced by uniform topology. Let R be a mapping from 
^([O,? 1 ],^ 1 ) into itself defined by 

R[ip]i(t) = A(Pit), te[0,T],iel. 

Given tp = (tp 1 ,^ 2 ) G P and ( G E, we define the dynamics associated with initial condition x 
and data ip,( as 

<Pi(t) = x i + y i t + ^(t)-R[ip 2 ] i (t) + ( i {t), i€l. (3.11) 

It is easy to see the analogy between the above equation and equation (|3.8p . and between the 
condition 9 ■ £ nondecreasing and property (|3. 10j) . The following condition will also be used, 

Pi(t) > 0, t > 0, i G X. (3.12) 

To define the game in the sense of Elliott and Kalton [SJ, we need the notion of strategies. A 
measurable mapping a : P — > E is called a strategy for the minimizing player if it satisfies the 
causality property. Namely, for every ^ = (-0 1 , V 2 ),^ = (^ 1 ,V' 2 ) G P and t G [0,T], 

(V' 1 ,-R[V' 2 ])(s) = (^ 1 ,-R[V' 2 ])(s) for all s G [0,t] implies a[V>](s) = a[^](s) for all s G [0,t]. 

(3.13) 

Given an initial condition x, a strategy a is said to be admissible if, for tp G P and £ = a[V>], 
the corresponding dynamics (|3.1ip satisfies the nonnegativity constraint (|3.12p . The set of all 
admissible strategies for the minimizing player is denoted by A. Given x and (V>, C) G -P x 
we define the cost by 

c(>, C) = / + ff(y(T)) - I(^), 

JO 

where tp is the corresponding dynamics given by (|3.1ip and I is given in Remark l3.3l The value 
of the game is defined by 

V{x) = inf sup c(?/>, a[^]). 

aeA x 

One can also obtain a simpler, equivalent formulation of the above game (see Remark 2.2 in 

El). 



JZ(Q n (0),B n ) = -logE 
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3.1.1 Main results 

Before we state our main results, let us introduce two conditions that will be used to prove 
the results. For to € R + , define 

h*(w) = m£{h(x) : x £ R\, 9 ■ x = w}, g*(w) = M{g(x) : x £ R\ , ■ x = w}. 

We impose the following condition. 

Condition 3.3 (Existence of a continuous minimizing curve) There exists a continuous 
map f : M+ — > such that for all w E R+, 

6-f(w) = w, h*(w) = h(f(w)), g*(w)=g(J(w)). 

We refer to PQ for the examples of h and g satisfying above condition. Similar condition is also 
used in [6], [3], where an analogous many-server model is treated in a diffusion regime. 

Condition 3.4 (Exponential moments) Denote At{iIj 1 ) = Yll=i su P[o,T] 1^(01- Then for 
any constant K, 

lim sup 4r log E^^")] < oo. 

In view of Proposition 2.1 in PQ, if there exists ao > such that supj 6 jE[e a ° /j4 '] < oo then 
Condition 13.41 holds. 

Remark 3.4 If Condition \3.4\ holds, then it is easy to see that for any constant K, 

limsupllogE[e fe ^(^(^) + ^^))] < oo. 

n— >oo O n 

Now we are ready to state our main results. 

Theorem 3.3 Let Conditions'^^ El and\3~4\ hold ondlim^oo = 0. Then lim V£(X n (0)) = 
V{x). 

Theorem 3.4 Let Conditions\£M El and\3~^ hold and'tim n -+ 00 = 0. Then lim V$(Q n (0)) = 
V{x). 

Theorem 3.5 Let Conditions \3.2 and \3.3\ hold. Ifh andg are bounded, then lim n _ ! . 00 V^(X n (0)) = 
V{x). 

3.2 Proof of Theorem S31 EH and [3751 
3.2.1 Lower bound 

Before we go in further details, let us mention a solution to the above game problem that 
was obtained in [I]. Recall the one-dimensional Skorohod map T from D([0,T],R) into itself. 
Given tp = (ip 1 ,-^ 2 ) G P, define 

4>(t) = x + yt + ^(t) - ip 2 (t), t€[0,T\. 
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We define 

aeim) = fmm)) - m, t g m, (z.u) 

where ipo[ip] = r\0 ■ tp]. Let us define ao[ip l ,ip 2 } = aoltp 1 , R[ip 2 ]]. In [1], it is proved that ag is 
a minimizing strategy for the game i.e., 

V(x) = sup c(ip, ag[ip]). (3.15) 

■4>eP 

For k > 0, we define 

D(k) = {V = (ip\i> 2 ) G D([0,T\,R 21 ) : ||^ 1 ||t + II^IIt < « and ^(0) G (3.16) 
where 

$(t) = x + yt + ^ 1 (t) — i2[^ 2 ](t), t€ [0,T]. 
It is shown in (pQ, (27)) that there exists constant 71,72 such that 

W&eMlt <7i(^t(V 1 ) + ^(V' 2 )+72, tG[0,T], (3.17) 

for all (ip l ,7p 2 ) G L>([0,T],]R 21 ). Given a map 99 : [0,T] -> IR fe and a constant 77 > 0, we define 
the 77-oscillation of 99 as 

osc v ((p) = sup{\\(p(s) - (p(t)\\ : \s - t\ < 7], s,t G [0,T]}. 

Then for any given k, e > there exists 5, 77 such that the followings hold: For any ip, ip G D(k) 

w&ew - a e m\\* T < e if nv^ 1 - ^ 2 ir + - ^ 2 ir < s, (3.18) 

and 

osc^ (&0 [V'] ) < e provided osc v (ip) < 5. (3.19) 

Theorem 3.6 Assume Conditions \£M and\3^E hold. Then liminf V£(X n (0)) > F(x). 

Proof: The proof of the theorem follows from [1] except some suitable modifications. We add 
here some details for clarity and convenience of the readers. Fix tp = (ip 1 ,^ 2 ) G P. Recall 
metric d(-, •) on D([0,T],R 21 ) which induces the J\ topology. Define, for r > 0, 

A r = {iP £ D([0,T],R 21 ) : d(^)<r}. 

Since ip is continuous, for any r\ G (0, 1) there exists r, 7] > such that 

tp G «4. r implies \\tp — ip\\* < r±, osCrj(ip) < r±. (3.20) 

This can be done as for any / G T (see Notations), 

-m\\ < \\m-Hfm\ + \mf(t))-m\\, 

\f(t)-t\* T < r( e w°-i), 
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and ip is uniformly continuous on [0, T]. Define 6 n = ( j^r^n , n™u u , ■ ■ ■ , N n^n ) ■ Then 6 r 
as n — > oo. Now, given < e < 1, choose a sequence of policies {-B n } C il n such that 

V^(X n (0)) + e > J x (X n (0), B n ) and B n G if 1 for all n. 



Recall 



where 



Xf (t) = (0) + y?t + i?(t) - ^(-^(t)) + Zf (t), (3.21) 



N n u n ^fn ■ /"* 

Z n {t) = _^±_ {p . t _ —!?($), T n (t) = J aft*)*. (3.22) 

We claim that for all n large and (A n , S™) G 

sup | (XJT | J < (6 + c)n, (3.23) 

where c = sup n ||y n ||. To prove the claim, let us take i G I such that |(X")~|y > (6 + c)r\ for 
infinitely many n. For large n, we have X™(0) > —r%. Hence we have times a™ < < T such 
that Xf(<7?) > -2ri, X?(a%) < -(c + 5)n and Xf(s) < ~n for all s G [<r?,o%]. Therefore 

^^bl^" - ^V'Jk^ - ~ ri for a11 s G [°"i' a 2]- Therefore using (I33TD we have 

N n u n n 

-(c + 3)rx > yfK - ) - 2o S c CT »_ CT » (A», 5;) + — ^— n(a 2 " - of). (3.24) 

Using (|3.20p and the fact — > oo, we see that (|3.24p leads to a contradiction for large n if 
(<T2 - of) > n. Again if (crZ? - of ) < n, then (I3.24j) is contradicting to (I3.20j) as the right 
most term in (|3.24p is non- negative. This proves the claim f)3.23|) . 
Given G > 0, define 

T n = M{t>0-.e n -Z n (t)>G}AT = mf[t>0 : ^(^^E^)) >G } AT ' 

" 8=1 

It is possible to choose K\ > such that for (A n , S 1 ") G A r and t > r n (see (38) in [1]), 

r ■ X n (i) > -ki + G. (3.25) 
Next, let ip : [0, T] — > M 1 be the dynamics corresponding to (ip,C)-, where ( = ao[ijj], namely 

<Pi(t) = Xl + Vi t + #(t) - R$ 2 ]i(t) + Ci(t). (3.26) 
To this end, we note that given k > 0, for large n (so that 9 n < 26) 

K an — 

e n . x > K ^e-x>- + — e-x~. (3.27) 

Then ip(t) = f(ipg[ip](t)) (I3.14|) . Let cdh [u> g ] be the modulus of continuity of h [resp. g] over 
{x G R 1 : a; • < We[^}\* T , Xi > -(c + 6)}. For k, § - 2\/7 sup n ||6» n ||(c + 6) > and 
large n we have 

M{h(x) : 6 n ■ x > K,Xi > -(c + 6)n} > inf{/i(x) : (9 ■ x > \ip e [tjj]\^,Xi > -(c + 6)n} 

> mf{h(x) : x £ R\, 9 ■ x > {(peMr} ~ &h(Vl(c + 6)n] 

> \h(<p)$-u h (y/l(c + 5)ri), 
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where for the first inequality we use (|3.27p and for the last inequality we use the monotonicity 
of h. Therefore we can find G > such that for all large n, (A n , 5") G A r , 

h(X n {t)) > \h{ip)\* - 5(n) and g{X n {t)) > g(<p(T)) - S(n), (3.28) 

on {t > r n } where S(ri) — > as r\ — > 0, and for t < r n 

\\Z n (t)\\ < k 2 . (3.29) 

for some constant &2 ■ Hence using (I3.21j) , (|3.20p and (I3.29|) , we obtain a constant K3 such that 
for (A n , S™) G A r and all n large 

sup \\XV- (t)\\ < « 3 . 
ie[o,T„] 

Consider the stochastic processes r,y n ,Z n , with values in M 1 such that (Xf)+(i) = ^ + 
+ Y"(t) - Y"(i) + Zf (t) on [0, Tn ] where 

Y"(t) = A?(tAr n ), 

Y?(t) = Xi - Xm - (A?)"(t) + ( W - y?)i + SMj^IW A r„)) - (1 - M?W A r„), 
Define VK n (t ) = s + yt + Y n (t ) - Y n (t ) + a e [Y n , Y n ] (t) . 

By ()3.29p . we have supj supr 0)Tn ] Ifti — ■^nTf , {t)\ — > as n — > 00. Therefore using the 
regularity property of ag (|3.19|) and (|3.2U|) . (|3.23|) with a proper choice of r\ < e we have 

y-W n \\* Tn <K 4 e. (3.30) 

for all n large and (A n ,S™) G A and some constant K4. Denote by Uh the modulus of 
continuity of /i [resp., g] over {(/ : ||q|| < \\tp\\^ + K3 + K4}. Hence using minimality property of 
olq we have 



f > h((X?)+)-u h (VI(c + 6)n) > M^ n W) -w h (x/7(c + 6)ri), 

\<?(X n (i)) > - w ff (\/7(c + 6)n) > ^(i)) - uj g {Vl{c + 6)n). 

for t G [0,t„] and G A for large n. 

Then by (f33DD and (ET5I1) . for (A n , S£) G A and all large n, 



(3.31) 



h(X n (s))ds > / /i(W n (s))ds-Tw h (V7(c + 6)e) 
j 

FT n 

> / h(ip{s))ds -Tu h (K A e) -Tu h (y/l(c + 6)e). 
Jo 

Combined with ()3.28|) this gives 

h(X n (s))ds> / h(ip(s))ds-T(u h (K 4 e)+cj h (VI(c + 6)e) + 5(e) 
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A similar argument gives 

g(X n (T)) = g(<p(T)) X{T < Tn} +g(v(T))x { T>T n } > g(<p(T))- (u g {K A e)+u g {Vl{c+Q)e) + 8{e) 
Hence using condition 13.21 it is easy to show that for all n large 



E[e b "[Jo T h(X n (s))ds+g(X n (T))]^ > E 



Jiltf h(X n (s))ds+g(X n (T))]„ 



bl ( lo h(v»W)*+fl(*»(T))-I(^)-o(e) 

> e V 

where o(e) = Toj h {n 4 e) + Tuj h {Vl(c + 6)e) + co g {VI(c + 6)e) + uj g (K 4 e) + (T + l)S(e). The proof 
follows letting n — > oo and e — >• 0. □ 

Theorem 3.7 Assume Conditions\£Band\£Mhold and lim = 0. Then liminf V§(X n (Q)) > 
V(x). 

Proof: We note that proof of Theorem 13.61 relies on two estimates, (|3,28p and (|3.30p . To get 
these estimates, we note that for any k > and large n, 9 n ■ X n > k + 1 =>• 9 ■ Q n > k/2 and 
\\X n - Q n \\* T = o(l). Hence the proof follows. □ 



3.2.2 Upper bound 



Theorem 3.8 Assume Conditions\£M El and{3m hold and lim f-^ = 0. Then lim sup V£(X n (0)) < 
V(x). 

Remark 3.5 If the functions h,g are bounded then Condition \'J.4\ is not required in the above 
statement. 



The proof is based on the construction of a suitable admissible policy. The main idea of the 
proof is similar to that appear in [1]. However, the proof appear here is complicated than that 
appear in pQ. The main difficulty we face here is due to the constrain on the policy that does 
not allow processor sharing. The idea is to make use of the preemptive behavior of the policy. 
We construct a policy that serves each class of customers over small time intervals (defined in 
suitable sense) and on average effort given to serve class i is ~ pi + (£ where the correction (£ 
is small and leads us to the correct limit. 



Proof: Let A > be a given constant. Define 

Q = {tp G Z?([0,T],M 21 ) : I(^) < A}. (3.32) 

By the definition of the rate function I (from Remark I3.3[) , Q is a compact set containing 
absolutely continuous paths starting from zero (particularly, QcP), with derivative having 
L 2 -norm uniformly bounded. Consequently, there exists a constant M = Ma such that \\ifi ||* + 
HV^H* < M for all tp £ Q. Consider the set D(M + 1) (I3T61) . let e £ (0, 1) be given, and choose 
5, r) > 0,5 G (0,e), as in (|3.18j) and (|3.19j) . corresponding to e and k = M + 1. It follows from 
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the 1? bound alluded to above, that for each fixed A, the members of Q are equicontinuous. 
Hence one can choose vq G (0, 77) (depending on A), such that 



osc„ (^) < — =, for all if> = (^ 1 ,V' 2 ) G Q, I = 1, 2, i G X. (3.33) 
4v 21 



Recall 

AW0 = {^e£>([o,r],R 2/ ) : d(V,^)<r}. 

Noting that, for any f £ T (see Notations), 

wm-mw < \\m-^f(t))\\ + Mf(t))-m\\, 

\f(t)-t\* T < T(eWr-l), 

it follows, by the equicontinuity of the members of Q, that it is possible to choose v\ > such 
that, for any ip G Q, 

^GA^WO implies ||V'-'0|r<j. (3.34) 

Let t>2 = mrn{uo,Vi, §}. Since Q is compact and I is lower semicontinuous, one can find a 
finite number of members tp 1 , 'ifj 2 , . . . , $ N of Q, and positive constants v 1 , . . . , v N with v k < V2, 
satisfying Q C U^A , and 

inf{I(^) : ^ G ^} > I(^ fe ) - |, fc = l,2,...,JV, (3.35) 

where, throughout, A k := ^4„fe(V5 fc ). 

Next we define a suitable policy such that the lower bound is asymptotically attained. 
Let v = Tj 1 A -j and L = L(u) = [f-J- Define a e = £ ■ jxp Then [c/,a^ +1 ) forms a disjoint 
partition of [0, T) with a = and sup, |c/ +1 — a | < u. Now consider a sequence {a n } such 
that "ftTC __s. as n — >• 00. Define ff n = |_^f~J where ft = t^ct- We can choose small 
enough so that inf n H n > 2. Define 

$ =a l + j — , j = 1, . . . , H n + 1. 

Hence , &n 5+1 ^)}fc(f f° rms a disjoint partition of [</, a e+1 ) where = a 1 , bn H = a e+l . 

Also 5 n := < a n for all n. Now we will split each [bii ,b$ +1 ^),£ = 1,2,... ,L + l,j = 

0, . . . , H n , using some random intervals. For this we define 

{?? = f B?(s)d8, 

A" = ~S% (w^Tl), (3.36) 
[P n = (A n ,D n ). 

We also denote £ = fix ■ 9) — x and 



b n a e [P n ]i(a e 1 ) - ae[P n ]i( 



l-2\ 



F?(cf) = V ->l-»j.v- 1 g > 2 (3>37) 
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9 ■ olq being nondecreasing we have Fp(a e ) > for all £ > 2. We need to define some more 
variable before we define the policy. We define random variables I > 2, as follows: 

mi) = { - ^"( a ')) + if Siez(Pi - < 1 and HPnll*^ < M + 2, 

* 1 pj otherwise. 

We also define for 



7< 



C„ K e t \+ -r ( n . fen k)+ < i 
Pi otherwise. 



Now we split the intervals ' = 0, 1, . . . , L, j = 0, 1, . . . , H n , using the above vari- 

ables as follows: For £ = 0, j = 0, . . . , H n , we define 

i 

= + J>£, i = l,2,...,I. 

k=i 

We fix the notation as cj(0) = b e J and cl j (I + 1) = for all £,j. For £ = 1, j = 0, . . . , F n , 

we define 

i 

= # +5 n Y J Pk, t = 1, 2, . . . , I. 

fc=l 

Finally, for £ > 2, j = 0, . . . , iF 1 , we define 



C ^)=*# + <^/f(£), i = l,2,...,I. 
fe=i 



It is easy to see that {[cn (i), Cn (i+l))}| = o forms a partition of [&n , &nP ) for all £ = 1, . . . , L+ 
1, j = 1, . . . , H n + 1. Now we are ready to define the policy. Recall from (j3.3|) . (|3.5p that 

For i EX, assume i?" is given by 

B?(t) = C?(*) A te[0,r), (3.39) 

where 

c n (t)= fjV ifiG[cl J (i-l),c^(i))forsome^j (g 4Q) 

1 otherwise. 

Now let us argue that B n is indeed an admissible policy. First we note that X n ,A n ,D n are 
piecewise constant processes. Since j n is deterministic, the policy is well defined on [0, a 2 ) with 
RCLL paths. This can be seen by applying induction on the jump times. Since £ > 2, 

is completely determined by the values of P n on [0, c/ _1 ], B n is uniquely defined on [0, T). Fix 
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Bf(T) = for all i G I. It is easy to check that B n satisfies all the requirement for being 
admissible control. Hence B n £ IP for all n. Hence by definition, 

^(X n (0)) < J%(X n (0),B n ). (3.41) 

With the policy defined above, we prove the result for upper bound. In what follows 
c\, C2, . . . , denote constants independent of A, e, 5, r], v and n. 

Define (p k {t) = where ^ k = x + yt + V M (*) - R[ip k > 2 ]{t). Recall from (IBTTl"]) 

that ip k is the dynamics corresponding to -ip k and ag[ip k ]. Let A n = AT{A n ) + At{S™) and 
denote by f2% the event {(A n ,S™) G .4 fc }. We prove the result in number of steps. First we 
show that for a constant ci, for all n > no(e,v), 

||*1t <ci(l + i„), (3.42) 

and 

sup||X n -/|| <cie, on J?£, fc = 1,2, . . . , JV. (3.43) 

Step 1: From (13471) and the fact j^T?(t) < t, we see that there exists C2 such that 

sup \\F n (a e )\\ < ■%?(! + llPnllt)- (3-44) 



a 'e[o,t] V™ « 

Since pi € (0, 1) for all i € X, we note from (|3.44j) that for all sufficiently large n, for any £ > 2, 

< M + 2 implies £fo - F l n {a l ))+ = ~ F>")) < 1, 

i i 

as £\ Ff (a*) > for all £ > 2. Define 

f„ = min{£ > 1 : ||P n ||*, > M + 2}. 

First we consider the event {f n = 1}. By definition, /3™(£) = Pi for all £ > 2 on {f n = 1}. 
For all large n, given i £ [6n , 6n ), for some £ = 0, 1, . . . , L, j = 0,1,..., H n , we have on 

{fn = 1}, 

< \fHa l - jf Cf{s)ds\ + | A (^' -a 1 )-^ £ Cf{s)ds\ (3.45) 
If ^ = 0, then the first term on the r.h.s. disappears and the second term is equal to 

iV n p v n ^ A 4 V n 



than equal to | fo^ lj \ ■ Hence using the fact that Cf < N n and \t — bj\ < 5 n we have 



If £ > 1, on {r n = 1}, second term on the r.h.s. of (|3,45p is equal to and the first term less 

3 using the fact that Cf < N n and \t 

sup ^\pit [ C?(s)ds\ < c 3 , 

E[0,T] On iV« ,/ 
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on {f n = 1} for all n large. Now we consider the event {f n > 1}. For t 6 [bn , &n J+1 ^), i < a Tn+1 , 
for some £ = 0, 1, . . . , L, j = 0, 1, . . . , H n , we have 

\Pit-^tiC?{s)ds\ 

< \Pi<S ~ Jo" C?(s)ds\ + WW C?{s)ds\ (3.46) 

+ \ Pi {t-b l i)-^il^CV-{s)ds\. 

Now if i < 1, then a similar argument as above holds to bound the r.h.s. of (|3.46p . So we 
consider £ > 2. Then for all n large 



1 



M - — / Cf{s)ds\ 



o 



i , /-a 1 />a^ /-6^ /■i 



< 



\c 3 + \ Pl (a e -a^-^N-v^2mk)\ 
+\ Pl (b ej - a e ) - ±-N n j5 n ^{k)\ + 25 n . 



N 

e-i 



< 



-^c 3 + \vY,F t n (a k )\ + ^)| + 

V ^ I n 



fc = 2 

&n , b 



< -^c 3 + |^(a e [P„](a^) - a e [P n ](0))\ + \j5 n F?{a*)\ + 28 n 

< ■^ = c 4 (l + ||P„|| t *) + 2<J n . 
y/n 

So now we are left with case t £ [b^ ,b^ +1 ^),t > a r " +1 , for some £ = 0,1,..., L, j 
0, 1, . . . , H n , on the event {f n > 1}. We note that, 



1 



\lHt - / C?{s)ds\ 

J 



-< v Jo ^ Ja f n + 1 



< -^=c 4 (l + ||P n || t *)+4£, 
fn 



s / — 

Hence combining all the calculations above and making use of the fact that " fc — > as 
n oo we have a constant C5 such that 

sup ^V--L j C?{s)ds\ <c 5 (l + I n ), (3.47) 
te[o,r] On A'" Jo 

for all n large. 
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Step 2: Now we are ready to prove (|3.42j) . Rewrite (|3.8j) as {Xf — e) + = Y™ + Z™, where 



z n (t) = £i / (Cf( S )-^( S ))+d S . 

1 w n b n N n J Q K 1 K ' 1 K " 

Now for each i £ I, (X™ — e) + is nonnegative and is nonnegative, nondecreasing. Since 
— > as n — > oo, for all n large (depending on e), 

{X?{s) - e)+ > => Xf (s) > ^iV n + Xf (a) > _V n =_> (Cf (s) - X"(s)) + = 0. 

Therefore J Q (Xf (s) -e) + dZ?(s) = 0. Therefore ((Xf-e)+, Z l n ) is the solution to the Skorohod 
problem for data Y£. Hence applying Lipschitz property of the Skorohod map and (I3.47D . we 
have 

\Z?\t + m - e) + lr < m\* T < cb(1 + A n ), (3.48) 
for all n large where we used the fact that (X? - e)~ < e + £^ < 1. Now El follows from 
dS3SD - Since 

______ n n J\J n 

using (|3.47|) . (|3.48|) and convergence of — , we have 

u n N n fin 1 

sup ^ ^-|p,t - — _ZT(*)| < c 7 (l + A n ), (3.49) 

t G [o,T] « o n N n 

for all n large. 

Step 3: In particular, (]3.49p implies that for all n large, 

sup \p i t--L;Tm<l (3-50) 
te[o,T] N 2 

holds on the event Ufc-7jP. Therefore using (|3.34p . (|3.33p and (|3.49p one obtains that for all 
large n (see (66) in [1]), 

sup \\S;(T n (t)) - R$*>*](t - < S -, (3.51) 

te[v,T] 2 

on _?£ , A; = 1, 2, . . . , N. In the rest of this step, we calculate the difference between Z n {t) (see 
(13. 9|) ) and [■?/;] (i — v) on the event _?£ . Recall f n defined above. We note that on one has 
f. n > L for all n large since H-P^It ^ PI* + ll^ll* < M + 2 by (jH3H) . Hence for all n large, 
on fl£,fc = l,2,...,-V, 

Tf = (pi - _^^) and ft(fi = (p i -F?(<S)), 
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for £ = 2, 3, . . . , L. Therefore for any t G [b°i , 6°^' +1 )), j = l,...,H n , we have 
n , 1 /"*,.. . t r , */ti . 1 „„ , . . 1 



< Hi-r—it - Jd n ) H : £j + Hi — n 

n V b n 



< 2^a n + ^^t4- (3-52) 

Now for k = 1,2, . . . , N , consider 

== Mi^(p<< " ^7 jl^{s)ds) - aim(t -v), t G [5,T], 

on the event J?*. We note from (j3.14j) that a e [^ fc ](0) = 1 Hence for t G [a 1 , a 2 ) (recall c/ = £v) 
and all large n, we have from (|3.33p and (|3.19p that 



mm < \w£ k (v) - u + \^(Pi(t - a 1 ) - ^ £ c* + \ii - 4$ k ut - v)\ 



where we use (|3.52p to estimate the first term and a similar estimate to calculate the second 
(for example, put £i = in (j3.52|) ) . Now we consider t G [a 2 ,T). Let t G [&n, &n J ) for some 
£ > 2, j = 0, . . . ,iJ n . The following calculations are of same type as step 1. We note that for 
large n, on i?^ - , 



= ^ (V - ^iV^( 7 f + * + E #(*))) + ^(pttf - #) - l^N n j5 n ^{£) 



n k=2 n 

= a e [P n U(£ - 2)5) + J -^[a e [P n ] t ((£ - 1)5) - a fl [P n ]i((* - 2)5)]. 

Therefore using l|Oty . (1339]) . (|5^5j> . (I3T341 and (l330|) . we see that for all n large, on fc 
1,2,... ,JV, 



|a e [P Tl ] i ((£-2)5)-a 9 [^ fc ] i (i--u)| < c 8 e 
|d(9[P n ]i((^-l)i5)-a e [P n ]i((^-2)t;)| < c 8 e 
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for some constant cs independent of t (see (68) in pQ ) . Hence combining all these calculations 

be fact that T~a n — > 

N n fj,f y/n/ 1 rt 



with (13.531) and using the fact that — > 0, we have, for all n large and all k 



sup 



n b n 



Pt-JriJ Cf{s)ds) - a e $%(t -v)\< c 9 e, (3.54) 



on fi k . 



Step 4: Recall <p k (t) = f(ipg[ip k ](t)). The goal of this step is to estimate the difference between 
X n and ip k on f2 k . To this end, let first 



p k (t) 



x + ^i for t E [0, v) 

f(ipe$ k ](t-v)) for te[v,T]. 



Also recall from step 2 that {Xf — e) + solves Skorohod problem for the date Y™. Since 
T((p k ) = (p k for all k = 1, . . . , N, we have for large n, 

\(X?-e)+-$\* T <2\Yi-$\* T . (3.55) 



Now for t G [0, i5), 



|fT(t)-#(*)l 

< |(Xf - e)- + X;(0) + yft + If (t) - S^JLj*®) 
+ tfN^Vn {pit _lf t C n {s)ds) _ Xi _ 



< c 10 s, 

for all n large where we use (|3.52p . ()3.33p and fl3.34f) . Similarly, using (I3.5ip . (I3.54|) . for 

te[v,T], 

\Yr(t)-${t)\ 

< \(X? - e)- + Xi(0) + y?t + i?(t) - §l{^r n (t)) - $>\t -v) + R[^%(t - v) 

..ijun /Z? 1 ft 

< cue. 



o 



Therefore from ()3.55j) . we get that for large n, on i?^ - , fe = 1, . . . , N, 

\X?-$\t<c 12 e. 

Thus (]3.43p follows by comparing (p k and </J fc . 

Rest of the proof follows by standard argument using (|3.42p and (|3.43|) (see for example, 
Step 5 in P). □ 



If b iV — )• 0, then — Q n ||^ — ► as n — > oo. Hence it is easy to obtain estimates like 

b„^/n 



(|3.42p and (|3.43|) for Q n with the policy constructed in Theorem 13.81 when h N "r- — > 0. Thus we 



have the following theorem: 
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400 b nv K 



0. Then 



Theorem 3.9 Let Conditions \3.2l HOI and \3J\ hold and lim 

limsup^(Cr(0))<y(x). 

n— >oo 

Theorem 3.10 Let Conditions \3.2\ and \3.3\ hold. If h and g are bounded, then 

limsu P y£(X n (0)) < V(x). 

n— >oo 

Proof: From Theorem l3.8l we see that we only need to consider the case when lim sup . /— > 0. 

Instead of introducing a new subsequence, we assume that lim b N ^ > 0. Hence lim b ^ n = 0. 

Given e G (0, 1), we construct an e— optimal policy. Since h, g are bounded, it is enough to 
construct a policy so that (|3.43j) holds for large n. 

Let A > be given. Define 

Q = {V>G D([0,T],R 21 ) : I(-0) < A}. (3.56) 

Hence Q C D(M) for a suitably chosen M. Using the same argument as in Theorem 13.81 we 
have v < e/2 such that 



osc v (V'') < 



, for all ^ = (V>\V> 2 ) G Q, / = 1,2, i GZ, 



4^21 



where 5 G (0, e) is chosen according to (|3.18p and 

ip G A v {iji) implies 



5 

<4' 



(3.57) 



(3.58) 



for all tp E Q. Also we can find finite number of members 
constants u 1 , . . . ,v N with t> fc < u, satisfying Q C U/c^4 fc , and 

e 



inf{I(^) : ip G A k } > I(V> 

where, throughout, ^4 fc := ^4„fc(Vi fc ). Define 

6 n a [P n ]jO'v) - ae[Pn]i((j - 1» 



, ip N of Q, and positive 
k = l,2,...,N, (3.59) 



*T(t) 



, for jv < t < (j + l)v, j > 1, (3.60) 



where P n = (A n ,D n ) (|3.36p . Since ag satisfies the causality property, F n is well defined. 
Denote 

Q(a,b) = axR+{a)x[o,i](b), a, b G R. 
Recall that £ = f(x ■ 0) — x. Define 



e(L(p, 



b f- 



Lp^ n Jx { x rw > £} AX™(t) 



) + )x { x» (t )> e} A*?(t) if i €[(),«) 



if t G [v, 2v) 



0(L(pi - ^ n (t - ^))iV n J , J>< - *T(t - ^)) + )x { x rW > £} A (t) otherwise. 

(3.61) 
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Using same argument as in Theorem 13.81 it is easy to see that B n is an admissible control 
and hence B n E il n . As earlier, define (p k (t) = f(ifg[ip k ](t)) where ^ k = x + yt + ip k,1 (t) — 
R[tp k ' 2 ](t). Denote by ft£ the event {(A n ,S^) e A k \. 



In what follows, c±, C2, . . . denote constants independent of Z\, e, n, v, 5, rj. 
As earlier (proof of Theorem I3.8[) , it is enough to show that there exists a constant c\ such 
for all n > no(e, i>), 



sup ||X n -/|| < cie, on fl* fc = 1, 2, . . . , iV. 



(3.62) 



First we note from ([3758]) that ||P n ||y < M + 2 on J2£ for k = 1, 2, . . . , N. Also from (I37T7D 
i 



and the fact -^-T™(i) < i, we see that there exists d,2 such that 



sup ||F"( S )||<^(l + ||P n || t *). 
se[v,t] V n v 

Therefore for all n large, pi ^7=17, Pi — P n (^ — £ I, are positive 



(3.63) 



and 



on J?£ , k = 1, 2, . . . , N where we use that 9 ■ £ = and Y2i Ft — 0- Again 

~ h 71 

Xf{t) > e X?(t) > Pl N n + eb n V^ e => X?(t) > ( Pi + ^J^W 



Hence using (|3.63p and the fact lim -3^ = 00, we have on f2£, k = 1, 2, . . . , N, 

X?(t) > e => X?(t) > ( Pi - F?(t - v))N n , t > 2v, 

for all n large. Similar fact holds in [0, v). Therefore for large n, we have B™(t) = YCf{t)\x^x n {t)>e} 
on k = 1, 2, . . . , N, where 



cm 



{pi 



PiN r 



h f 

n 1 \ Am 



PiV n v 



)N n ift€[0,v) 



if t £ [v, 2v) 



(3.64) 



k (jH - F?(t - v))N n otherwise. 
Using the same argument as (|3.23p . we have for large n, 

S up\(X?)-\* T <(c + 6)e, 



(3.65) 



on J?£, k = 1, 2, . . . , N where c = sup n ||y n ||. Following the same arguments in (Step 1, [T]) we 
obtain, for large n, 



sup ^\pi - -j- / C?(s)ds\ < c 4 , 
[0>T] b n N n J 



(3.66) 
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on Q 7 ^ for some constant C4. We rewrite (|3.8p as (Xf — e) + = + Zf where 



Since (X" — e) + > =>■ X{x«( s )< £ } = 0, (Xf — e) + solves Skorohod problem for the data Y™. 
Hence on QT, for large n, supj \Z™\t < C5 using the fact that 



sup '^ ~k f ~ LCTMJ)*I < 0. 

[ 0)T] b n N n J b n N n 

Combining with ()3.66|) . for large n, 

sup sup -r—\pi - -j-r- / Sf (s)ds| < c 6 , (3.67) 
i [0,T] »n iV n Jo 

on for some constant cq. Now we can use the same arguments as in (Step 3, [1]) to conclude 
that for large n, on {2%, 



sup | —^(pit - — / Bf - a^ fc ](t - «)| < c 7 e, (3.68) 



for some constant cy. We define (/5 fc as in Step 4 above replacing v by v. Using the Lipschitz 
property of the Skorohod map, we obtain on {7g, for large n, 

\(Xr) + -^\* T <2\Y l n -^\* T <c 8 e, 

where the last estimate is obtained using the same argument as Step 4 above. Combining with 
(|3.65p . we have on {2%, k = 1, 2, . . . ,N, \Xf — <p^\j< < cge for some constant eg and n large. 
Hence we obtain (|3.62p comparing (p and (p. □ 

3.3 Linear cost and asymptotic optimality 

In this section, we provide a simple policy based on priority that is asymptotically optimal. 
We assume that h and g have following forms 

1 1 

h(x) = axi, g{x) = diXi, 

i=l i=l 

where q and are nonnegative constants, and, in addition, 

c\pi > C2/^2 > ■ • • > ciMl and d\p\ > (I2P2 > ■ • ■ > ^1/^1- 
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We consider the c/x-rule that prioritizes according to the ordering of class labels, with highest 
priority to class 1. Define 

B™ = X?, B% = A 2 n A (N n — B"), . . . , B? = X? A {N n - ^ B ?)- ( 3 - 69 ) 

It is easy to see that the above policy is consistent with f)3.3j) - (|3.5[) and B n 6 il n . Proof of the 
following theorem follows using the same argument from Theorem 5.1 in [1 . 

Theorem 3.11 Assume Conditions ] \3Ji\ hold and lim b N ™^ = 0- Then, under the priority 
policy {B n } of (1339]) . 

lim jP ) (Q n (0),B n ) = lim J%(X n (0), B n ) = V(x). 



Acknowledgement: The author is grateful to Prof. Rami Atar for valuable discussions. 

References 

[1] R. Atar and A. Biswas. Control of the multiclass G/G/l queue in the moderate deviation 
regime. Submitted, 2012. 

[2] R. Atar, P. Dupuis, and A. Shwartz. An escape-time criterion for queueing networks: 
asymptotic risk-sensitive control via differential games. Math. Oper. Res., 28(4):801-835, 
2003. 

[3] R. Atar, A. Goswami, and A. Shwartz. Risk-sensitive control for the parallel server model 
and an exponential version of the c/i rule. Submitted, 2011. 

[4] R. Atar and I. Gurvich. Scheduling parallel servers in the non-degenerate slowdown 
diffusion regime: Asymptotic optimality results. Submitted, 2011. 

[5] R. Atar, A. Mandelbaum, and M. I. Reiman. Scheduling a multi class queue with many 
exponential servers: asymptotic optimality in heavy traffic. Ann. Appl. Probab., 14(3): 
1084-1134, 2004. 

[6] R. Atar and N. Solomon. Asymptotically optimal interruptible service policies for schedul- 
ing jobs in a diffusion regime with nondegenerate slowdown. Queueing Systems Theory 
Appl, 69(217-235), 2011. 

[7] P. Billingsley. Convergence of probability measures. Wiley Series in Probability and Statis- 
tics: Probability and Statistics. John Wiley & Sons Inc., New York, second edition, 1999. 
ISBN 0-471-19745-9. x+277 pp. A Wiley-Interscience Publication. 

[8] H. Chen and A. Mandelbaum. Leontief systems, RBVs and RBMs. In Applied stochas- 
tic analysis (London, 1989), volume 5 of Stochastics Monogr., pages 1-43. Gordon and 
Breach, New York, 1991. 



28 



[9] R. J. Elliott and N. J. Kalton. The existence of value in differential games. Ameri- 
can Mathematical Society, Providence, R.I., 1972. iv+67 pp. Memoirs of the American 
Mathematical Society, No. 126. 

[10] A. Ganesh, N. O'Connell, and D. Wischik. Big queues, volume 1838 of Lecture Notes in 
Mathematics. Springer- Verlag, Berlin, 2004. ISBN 3-540-20912-3. xii+254 pp. 

[11] J. Garcia. An extension of the contraction principle. J. Theoret. Probab., 17(2):403-434, 
2004. 

[12] S. Halfin and W. Whitt. Heavy-traffic limits for queues with many exponential servers. 
Oper. Res., 29(3):567-588, 1981. 

[13] J. M. Harrison and M. I. Reiman. Reflected Brownian motion on an orthant. Ann. Probab., 
9(2):302-308, 1981. 

[14] J. Lynch and J. Sethuraman. Large deviations for processes with independent increments. 
Ann. Probab., 15(2):610-627, 1987. 

[15] K. Majewski. Sample path large deviations for multiclass feedforward queueing networks 
in critical loading. Ann. Appl. Probab., 16(4): 1893-1924, 2006. 

[16] A. A. Puhalskii. Moderate deviations for queues in critical loading. Queueing Systems 
Theory Appl, 31(3-4):359-392, 1999. 

[17] A. A. Puhalskii and W. Whitt. Functional large deviation principles for first-passage-time 
processes. Ann. Appl. Probab., 7(2):362-381, 1997. 

[18] P. Whittle. Risk- sensitive optimal control. Wiley-Interscience Series in Systems and Op- 
timization. John Wiley & Sons Ltd., Chichester, 1990. ISBN 0-471-92622-1. x+246 pp. 

[19] D. Wischik. Moderate deviations in queueing theory. Unpublished manuscript, 2001. 



29 



